Skip to content

fix(ci): split Playwright install to prevent hang on GitHub Actions#152

Merged
constantinius merged 1 commit into
mainfrom
constantinius/fix/ci/playwright-install-hang
Jun 15, 2026
Merged

fix(ci): split Playwright install to prevent hang on GitHub Actions#152
constantinius merged 1 commit into
mainfrom
constantinius/fix/ci/playwright-install-hang

Conversation

@constantinius

Copy link
Copy Markdown
Collaborator

Problem

The daily test run has been broken since May 29 — every run times out at the 6-hour GitHub Actions job limit. All 18 consecutive daily runs have failed.

Root cause: npx playwright install --with-deps chromium hangs indefinitely after the Chrome download completes (reaches 100% but never finishes extraction/setup). This started when the GitHub Actions runner image updated, likely causing an interaction issue between --with-deps system dependency installation (needrestart/dpkg) and the browser download process.

Ref: https://github.com/getsentry/testing-ai-sdk-integrations/actions/runs/27492154452

Fix

Split the combined install into two separate steps:

  1. npx playwright install-deps chromium — installs system dependencies only, with DEBIAN_FRONTEND=noninteractive and NEEDRESTART_MODE=a to prevent any interactive prompts
  2. npx playwright install chromium — downloads and installs browser binaries

Both steps have a 5-minute timeout as a safety net.

Applied to both daily-tests.yml and pr-tests.yml workflows.

The combined 'npx playwright install --with-deps chromium' command has
been hanging after Chrome download completes on newer GitHub Actions
runner images since May 29, causing every daily test run to time out
at the 6-hour job limit.

Split into two separate steps:
- 'npx playwright install-deps chromium' for system dependencies
  (with DEBIAN_FRONTEND=noninteractive and NEEDRESTART_MODE=a)
- 'npx playwright install chromium' for browser binaries

Both steps have a 5-minute timeout to fail fast if they hang again.
Applied to both daily-tests.yml and pr-tests.yml workflows.
@constantinius constantinius merged commit 366d8e2 into main Jun 15, 2026
12 of 14 checks passed
@constantinius constantinius deleted the constantinius/fix/ci/playwright-install-hang branch June 15, 2026 06:12
@github-actions

Copy link
Copy Markdown

🟡 AI SDK Integration Test Results

Status: 1 test fixed, 464 still failing

Summary

Metric main PR Change
Total Tests 847 847
Passed 246 247 +1 ✅
Failed 465 464 -1 ✅

✅ Fixed

These tests were failing on main but are now passing:

  • node/vercel :: Conversation ID Agent Test (blocking, function, anthropic)

Test Matrix

Agent Tests

SDK Basic Agent Test Conversation ID Agent Test Long Input Agent Test Tool Call Agent Test Tool Error Agent Test Vision Agent Test
cloudflare/langgraph anthropicopenai anthropicopenai anthropicopenai anthropicopenai anthropicopenai anthropicopenai
cloudflare/vercel blk, class, anthropicblk, class, openaiblk, function, anthropicblk, function, openaistr, class, anthropicstr, class, openaistr, function, anthropicstr, function, openai blk, class, anthropicblk, class, openaiblk, function, anthropicblk, function, openaistr, class, anthropicstr, class, openaistr, function, anthropicstr, function, openai blk, class, anthropicblk, class, openaiblk, function, anthropicblk, function, openaistr, class, anthropicstr, class, openaistr, function, anthropicstr, function, openai blk, class, anthropicblk, class, openaiblk, function, anthropicblk, function, openaistr, class, anthropicstr, class, openaistr, function, anthropicstr, function, openai blk, class, anthropicblk, class, openaiblk, function, anthropicblk, function, openaistr, class, anthropicstr, class, openaistr, function, anthropicstr, function, openai blk, class, anthropicblk, class, openaiblk, function, anthropicblk, function, openaistr, class, anthropicstr, class, openaistr, function, anthropicstr, function, openai
nextjs/mastra
nextjs/vercel blk, class, anthropicblk, class, openaiblk, function, anthropicblk, function, openaistr, class, anthropicstr, class, openaistr, function, anthropicstr, function, openai blk, class, anthropicblk, class, openaiblk, function, anthropicblk, function, openaistr, class, anthropicstr, class, openaistr, function, anthropicstr, function, openai blk, class, anthropicblk, class, openaiblk, function, anthropicblk, function, openaistr, class, anthropicstr, class, openaistr, function, anthropicstr, function, openai blk, class, anthropicblk, class, openaiblk, function, anthropicblk, function, openaistr, class, anthropicstr, class, openaistr, function, anthropicstr, function, openai blk, class, anthropicblk, class, openaiblk, function, anthropicblk, function, openaistr, class, anthropicstr, class, openaistr, function, anthropicstr, function, openai blk, class, anthropicblk, class, openaiblk, function, anthropicblk, function, openaistr, class, anthropicstr, class, openaistr, function, anthropicstr, function, openai
node/langgraph anthropicopenai anthropicopenai anthropicopenai anthropicopenai anthropicopenai anthropicopenai
node/manual
node/mastra
node/vercel blk, class, anthropicblk, class, openaiblk, function, anthropicblk, function, openaistr, class, anthropicstr, class, openaistr, function, anthropicstr, function, openai blk, class, anthropicblk, class, openai ✅🔧blk, function, anthropicblk, function, openaistr, class, anthropicstr, class, openaistr, function, anthropicstr, function, openai blk, class, anthropicblk, class, openaiblk, function, anthropicblk, function, openaistr, class, anthropicstr, class, openaistr, function, anthropicstr, function, openai blk, class, anthropicblk, class, openaiblk, function, anthropicblk, function, openaistr, class, anthropicstr, class, openaistr, function, anthropicstr, function, openai blk, class, anthropicblk, class, openaiblk, function, anthropicblk, function, openaistr, class, anthropicstr, class, openaistr, function, anthropicstr, function, openai blk, class, anthropicblk, class, openaiblk, function, anthropicblk, function, openaistr, class, anthropicstr, class, openaistr, function, anthropicstr, function, openai
python/langgraph a, anthropica, openais, anthropics, openai a, anthropica, openais, anthropics, openai a, anthropica, openais, anthropics, openai a, anthropica, openais, anthropics, openai a, anthropica, openais, anthropics, openai a, anthropica, openais, anthropics, openai
python/manual as as as as as as
python/openai-agents
python/pydantic-ai a, fallbacka, single a, fallbacka, single a, fallbacka, single a, fallbacka, single a, fallbacka, single a, fallbacka, single

Embedding Tests

SDK Basic Embeddings Test
browser/google-genai
browser/langchain
browser/openai
cloudflare/google-genai
cloudflare/langchain
cloudflare/openai
cloudflare/vercel
nextjs/google-genai
nextjs/langchain
nextjs/openai
nextjs/vercel
node/google-genai
node/langchain
node/openai
node/vercel
python/google-genai a, blks, blk
python/langchain a, blks, blk
python/litellm a, blks, blk
python/manual a, blks, blk
python/openai a, blks, blk

LLM Tests

SDK Basic Error LLM Test Basic LLM Test Conversation ID LLM Test Long Input LLM Test Multi-Turn LLM Test Vision LLM Test
browser/anthropic blkstr blkstr blkstr blkstr blkstr blkstr
browser/google-genai blkstr blkstr blkstr blkstr blkstr blkstr
browser/langchain blk, anthropicblk, openaistr, anthropicstr, openai blk, anthropicblk, openaistr, anthropicstr, openai blk, anthropicblk, openaistr, anthropicstr, openai blk, anthropicblk, openaistr, anthropicstr, openai blk, anthropicblk, openaistr, anthropicstr, openai blk, anthropicblk, openaistr, anthropicstr, openai
browser/openai blk, chatblk, responsesstr, chatstr, responses blk, chatblk, responsesstr, chatstr, responses blk, chatblk, responsesstr, chatstr, responses blk, chatblk, responsesstr, chatstr, responses blk, chatblk, responsesstr, chatstr, responses blk, chatblk, responsesstr, chatstr, responses
cloudflare/anthropic blkstr blkstr blkstr blkstr blkstr blkstr
cloudflare/google-genai blkstr blkstr blkstr blkstr blkstr blkstr
cloudflare/langchain blk, anthropicblk, openaistr, anthropicstr, openai blk, anthropicblk, openaistr, anthropicstr, openai blk, anthropicblk, openaistr, anthropicstr, openai blk, anthropicblk, openaistr, anthropicstr, openai blk, anthropicblk, openaistr, anthropicstr, openai blk, anthropicblk, openaistr, anthropicstr, openai
cloudflare/openai blk, chatblk, responsesstr, chatstr, responses blk, chatblk, responsesstr, chatstr, responses blk, chatblk, responsesstr, chatstr, responses blk, chatblk, responsesstr, chatstr, responses blk, chatblk, responsesstr, chatstr, responses blk, chatblk, responsesstr, chatstr, responses
nextjs/anthropic blkstr blkstr blkstr blkstr blkstr blkstr
nextjs/google-genai blkstr blkstr blkstr blkstr blkstr blkstr
nextjs/langchain blk, anthropicblk, openaistr, anthropicstr, openai blk, anthropicblk, openaistr, anthropicstr, openai blk, anthropicblk, openaistr, anthropicstr, openai blk, anthropicblk, openaistr, anthropicstr, openai blk, anthropicblk, openaistr, anthropicstr, openai blk, anthropicblk, openaistr, anthropicstr, openai
nextjs/openai blk, chatblk, responsesstr, chatstr, responses blk, chatblk, responsesstr, chatstr, responses blk, chatblk, responsesstr, chatstr, responses blk, chatblk, responsesstr, chatstr, responses blk, chatblk, responsesstr, chatstr, responses blk, chatblk, responsesstr, chatstr, responses
node/anthropic blkstr blkstr blkstr blkstr blkstr blkstr
node/google-genai blkstr blkstr blkstr blkstr blkstr blkstr
node/langchain blk, anthropicblk, openaistr, anthropicstr, openai blk, anthropicblk, openaistr, anthropicstr, openai blk, anthropicblk, openaistr, anthropicstr, openai blk, anthropicblk, openaistr, anthropicstr, openai blk, anthropicblk, openaistr, anthropicstr, openai blk, anthropicblk, openaistr, anthropicstr, openai
node/manual
node/openai blk, chatblk, responsesstr, chatstr, responses blk, chatblk, responsesstr, chatstr, responses blk, chatblk, responsesstr, chatstr, responses blk, chatblk, responsesstr, chatstr, responses blk, chatblk, responsesstr, chatstr, responses blk, chatblk, responsesstr, chatstr, responses
python/anthropic a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str
python/google-genai a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str
python/langchain a, blk, anthropica, blk, openaia, str, anthropica, str, openais, blk, anthropics, blk, openais, str, anthropics, str, openai a, blk, anthropica, blk, openaia, str, anthropica, str, openais, blk, anthropics, blk, openais, str, anthropics, str, openai a, blk, anthropica, blk, openaia, str, anthropica, str, openais, blk, anthropics, blk, openais, str, anthropics, str, openai a, blk, anthropica, blk, openaia, str, anthropica, str, openais, blk, anthropics, blk, openais, str, anthropics, str, openai a, blk, anthropica, blk, openaia, str, anthropica, str, openais, blk, anthropics, blk, openais, str, anthropics, str, openai a, blk, anthropica, blk, openaia, str, anthropica, str, openais, blk, anthropics, blk, openais, str, anthropics, str, openai
python/litellm a, blk, completion, anthropica, blk, completion, openaia, blk, responses, anthropica, blk, responses, openaia, str, completion, anthropica, str, completion, openaia, str, responses, anthropica, str, responses, openais, blk, completion, anthropics, blk, completion, openais, blk, responses, anthropics, blk, responses, openais, str, completion, anthropics, str, completion, openais, str, responses, anthropics, str, responses, openai a, blk, completion, anthropica, blk, completion, openaia, blk, responses, anthropica, blk, responses, openaia, str, completion, anthropica, str, completion, openaia, str, responses, anthropica, str, responses, openais, blk, completion, anthropics, blk, completion, openais, blk, responses, anthropics, blk, responses, openais, str, completion, anthropics, str, completion, openais, str, responses, anthropics, str, responses, openai a, blk, completion, anthropica, blk, completion, openaia, blk, responses, anthropica, blk, responses, openaia, str, completion, anthropica, str, completion, openaia, str, responses, anthropica, str, responses, openais, blk, completion, anthropics, blk, completion, openais, blk, responses, anthropics, blk, responses, openais, str, completion, anthropics, str, completion, openais, str, responses, anthropics, str, responses, openai a, blk, completion, anthropica, blk, completion, openaia, blk, responses, anthropica, blk, responses, openaia, str, completion, anthropica, str, completion, openaia, str, responses, anthropica, str, responses, openais, blk, completion, anthropics, blk, completion, openais, blk, responses, anthropics, blk, responses, openais, str, completion, anthropics, str, completion, openais, str, responses, anthropics, str, responses, openai a, blk, completion, anthropica, blk, completion, openaia, blk, responses, anthropica, blk, responses, openaia, str, completion, anthropica, str, completion, openaia, str, responses, anthropica, str, responses, openais, blk, completion, anthropics, blk, completion, openais, blk, responses, anthropics, blk, responses, openais, str, completion, anthropics, str, completion, openais, str, responses, anthropics, str, responses, openai a, blk, completion, anthropica, blk, completion, openaia, blk, responses, anthropica, blk, responses, openaia, str, completion, anthropica, str, completion, openaia, str, responses, anthropica, str, responses, openais, blk, completion, anthropics, blk, completion, openais, blk, responses, anthropics, blk, responses, openais, str, completion, anthropics, str, completion, openais, str, responses, anthropics, str, responses, openai
python/manual a, blks, blk a, blks, blk a, blks, blk a, blks, blk a, blks, blk
python/openai a, blk, chata, blk, responsesa, str, chata, str, responsess, blk, chats, blk, responsess, str, chats, str, responses a, blk, chata, blk, responsesa, str, chata, str, responsess, blk, chats, blk, responsess, str, chats, str, responses a, blk, chata, blk, responsesa, str, chata, str, responsess, blk, chats, blk, responsess, str, chats, str, responses a, blk, chata, blk, responsesa, str, chata, str, responsess, blk, chats, blk, responsess, str, chats, str, responses a, blk, chata, blk, responsesa, str, chata, str, responsess, blk, chats, blk, responsess, str, chats, str, responses a, blk, chata, blk, responsesa, str, chata, str, responsess, blk, chats, blk, responsess, str, chats, str, responses

MCP Tests

SDK Basic MCP Tool Call Test MCP Multiple Tool Calls Test MCP Prompt Get Test MCP Resource Read Test MCP Tool Error Test
node/mcp sseio sseio sseio sseio sseio
python/fastmcp a, blk, ssea, blk, io a, blk, ssea, blk, io a, blk, ssea, blk, io a, blk, ssea, blk, io a, blk, ssea, blk, io
python/mcp a, blk, sse, hia, blk, sse, loa, blk, io, hia, blk, io, lo a, blk, sse, hia, blk, sse, loa, blk, io, hia, blk, io, lo a, blk, sse, hia, blk, sse, loa, blk, io, hia, blk, io, lo a, blk, sse, hia, blk, sse, loa, blk, io, hia, blk, io, lo a, blk, sse, hia, blk, sse, loa, blk, io, hia, blk, io, lo

Legend: ✅ Pass | ❌ Fail | ✅🔧 Fixed | ❌📉 Regressed | ✅🆕 New (pass) | ❌🆕 New (fail) | 🗑️ Removed | str=streaming blk=blocking a=async s=sync io=stdio sse=sse hi=highlevel lo=lowlevel


Generated by AI SDK Integration Tests

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant